1. Field of the invention
The present invention relates to a 3D depth imaging method and system, and more particularly, to a 3D depth imaging method and system capable of more exactly imaging a 3D depth by exactly restoring a modified signal using a new scheme called signal separation coding.
2. Description of the Prior Art
In general, a method of three-dimensional (3D) depth imaging using a structural light is recently receiving attention because it is suitable for sensing a 3D environment in service robotics. The basic principle of depth imaging using a structural light, which is an active stereo scheme, is to radiate light at an object using a projection means such as a projector, image the object irradiated with light using an image receiving means such as a camera, and observe the extent of distortion of the light due to the object in order to calculate the depth of the object and obtain a depth image.
FIG. 1 is a schematic diagram illustrating a principle of a structural light-based 3D depth imaging system. As shown in FIG. 1, a 3D position of one point x on an object 100 is determined as an intersection point of a straight line connected between an origin Op of a projection means and a point p on the retinal plane 200 of the projection means, and a straight line connected between an origin Oc of an image receiving means and a point q on a retinal plane 300 of the image receiving means. Accordingly, a depth image can be obtained by calculating the coordinates of a point x as a pair of address values on each retinal plane at the points p and q after the projector and the camera are calibrated. That is, the core of the depth imaging method employing such a stereo scheme is to determine a pixel correspondence point between the received image and the projected image. With the determined correspondence point, the depth can be easily calculated using simple geometry.
For the accuracy of depth imaging, a light pattern projected by the projection means is coded spatially and/or temporally according to a time sequence on a pixel array so that a spatial and/or temporal address of the signal detected by the image receiving means uniquely determines the pixel correspondence point of the corresponding projection means.
Examples of such a conventional coding method include direct coding, spatial coding, temporal coding, and hybrid coding methods.
In the direct coding method, a grey and/or color level is directly used in coding, and a disparity image is calculated through one pattern frame. This method has the advantage of high speed. However, the accuracy and robustness of the method are poor due to ambient illumination variation and noise.
In the spatial coding method, a specially designed, coded pattern, which is arranged on a pixel array, is used. A De Brujin sequence, a quasi random code or the like is also used. The thus coded pattern is used to provide pixel address information from an adjacent pixel. Spatial coding obtains a disparity image from one or two frame patterns. This method has high speed and improved robustness in error correction because of its address information. However, spatial coding is affected by signal modification or complexity of an object because it uses spatially arranged address information for pixel correspondence.
Temporal coding uses coding patterns arranged along the time axis. A binary code, an N-ary code, and a line shifting-based gray code have been suggested. Generally, temporal coding has higher accuracy than spatial coding. This is because temporal coding has no order restriction and is possible using a black and white color signal. However, temporal coding is not suitable for a rapidly changing scene because of its use of a frame sequence.
Hybrid coding uses a mixture of temporal coding and spatial coding. This coding method can obtain a robust depth image. However, hybrid coding cannot be used in a very complex environment because it has the shortcomings of spatial coding. Spatial coding and hybrid coding are suitable for an object having a continuous surface. They may undergo transposition of the address sequence in a discontinuous surface.
Such conventional coding methods are focused on designing a spatial and/or time code in calculating a single pixel based on address information. However, the conventional coding methods have a fundamental limitation in calculating a higher-accuracy pixel correspondence point needed for exact depth imaging. In particular, they overlook a complex boundary surface such as occluding and shading the neighborhood of a boundary, or transposition that address information undergoes in pixel correspondence. Accordingly, the accuracy of depth imaging for a complex object is very low. It results from the fact that the conventional method is focused on coding the received signal based on address information and does not consider a signal modification extent making the address information inexact at the image receiving means side. Further, even though a long time code sequence providing each pixel at a projection means side having a unique address is used, it is not effective in processing signal modification.
Further, it should be considered that the signal received by the image receiving means is affected by system/environmental noise such as light scattering, reflectance change, and ambient illumination variation, and undergoes multiple code mixing in which the signal is mixed with pixels adjacent to the projection means and even remote pixels. However, conventional methods overlook such matters.
Accordingly, the present inventors suggest a new coding method called a “signal separation scheme” to solve the aforementioned problems.